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Abstract 

In many applications it is important to be able to sample paths of SDEs conditional 
on observations of various kinds. This paper studies SPDEs which solve such sam- 
pling problems. The SPDE may be viewed as an infinite dimensional analogue of the 
Langevin SDE used in finite dimensional sampling. Here the theory is developed for 
conditioned Gaussian processes for which the resulting SPDE is linear. Applications 
include the Kalman-Bucy filter/smoother. A companion paper studies the nonlinear 
case, building on the linear analysis provided here. 



1 Introduction 

An important basic concept in sampling is Langevin dynamics: suppose a target den- 
sity p on M. d has the form p(x) = cexp(— V(x)). Then the stochastic differential 
equation (SDE) 

dx r dW 

- = -W(,) + V2— (1.1) 

has p as its invariant density. Thus, assuming that ( II. Il l is ergodic, x(t) produces sam- 
ples from the target density p as t — > oo. (For details see, for example, |RC99 1.) 

In |SVW04| we give an heuristic approach to generalising the Langevin method 
to an infinite dimensional setting. We derive stochastic partial differential equations 
(SPDEs) which are the infinite dimensional analogue of i ll . It . These SPDEs sample 
from paths of stochastic differential equations, conditional on observations. Observa- 
tions which can be incorporated into this framework include knowledge of the solution 
at two points (bridges) and a set-up which includes the Kalman-Bucy filter/smoother. 
For bridge sampling the SPDEs are also derived in |RVE05 1, their motivation being to 
understand the invariant measures of SPDEs through bridge processes. 

In the current paper we give a rigorous treatment of this SPDE based sampling 
method when the processes to be sampled are linear and Gaussian. The resulting 
SPDEs for the sampling are also linear and Gaussian in this case. We find it useful 
to present the Gaussian theory of SPDE based sampling for conditioned diffusions in a 
self-contained fashion for the following reasons. 
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• For nonlinear problems the SPDE based samplers can be quite competitive. A 
companion article |HSV| will build on the analysis in this paper to analyse 
SPDEs which sample paths from nonlinear SDEs, conditional on observations. 
The mathematical techniques are quite different from the Gaussian methods used 
here and hence we present them in a separate paper. However the desired path- 
space measures there will be characterised by calculating the density with respect 
to the Gaussian measures calculated here. 

• We derive an explicit description of the Kalman/Bucy smoother via the solution 
of a linear two-point boundary value problem. This is not something that we 
have found in the existing literature; it is strongly suggestive that for off-line 
smoothing of Gaussian processes there is the potential for application of a range 
of fast techniques available in the computational mathematics literature, and dif- 
ferent from the usual forward/backward implementation of the filter/smoother. 
See section|4] 

• For Gaussian processes, the SPDEs studied here will not usually constitute the 
optimal way to sample, because of the time correlation inherent in the SPDE; 
better methods can be developed to generate independent samples by factoris- 
ing the covariance operator. However these better methods can be viewed as a 
particular discretisation of the SPDEs written down in this paper, and this con- 
nection is of both theoretical interest and practical use, including as the basis for 
algorithms in the nonlinear case. See section|5]and [?]. 

In section |21 of this article we will develop a general MCMC method to sample 
from a given Gaussian process. It transpires that the distribution of a centred Gaussian 
process coincides with the invariant distribution of the L 2 -valued SDE 



where C is the inverse of the covariance operator, m is the mean of the process and w 
is a cylindrical Wiener process. 

The first sampling problems we consider are governed by paths of the M^-valued 
linear SDE 



subject to observations of the initial point X(0), as well as possibly the end-point X(l). 
Here we have A, B £ M. dxd and W is a standard d-dimensional Brownian motion. 
Since the SDE is linear, the solution X is a Gaussian process. Section 3 identifies 
the operator C in the case where we sample solutions of II. 31 . subject to end-point 
conditions. In fact, £ is a second order differential operator with boundary conditions 
reflecting the nature of the observations and thus we can write ll.2t as an SPDE. 

In section 4 we study the situation where two processes X and Y solve the linear 
system of SDEs 




Vt S (0,oo), 



(1.2) 




Vu e [0, 1] 



(1.3) 



—(u) = A 11 X(u) + B 11 
du 

dY 

— («) = A 21 X(u) + B 22 



dW 3 



X 



(«) 



du 
dW x 



y 



(«) 



du 



on [0, 1] and we want to sample paths from the distribution of X (the signal) condi- 
tioned on Y (the observation). Again, we identify the operator C in II. 21 as a second 



Gaussian Processes 



3 



order differential operator and derive an SPDE with this distribution as its invariant 
distribution. We also give a separate proof that the mean of the invariant measure of 
the SPDE coincides with the standard algorithmic implementation of the Kalman-Bucy 
filter/smoother through forward/backward sweeps. 

Section|5]contains some brief remarks concerning the process of discretising SPDEs 
to create samplers, and section[6]contains our conclusions. 

To avoid confusion we use the following naming convention. Solutions to SDEs 
like J 1 .31 which give our target distributions are denoted by upper case letters. So- 
lutions to infinite dimensional Langevin equations like II. 2> which we use to sample 
from these target distributions are denoted by lower case letters. 

2 Gaussian Processes 

In this section we will derive a Hilbert space valued SDE to sample from arbitrary 
Gaussian processes. 

Recall that a random variable X taking values in a separable Hilbert space TL is 
said to be Gaussian if the law of (y, X) is Gaussian for every y E TL (Dirac measures 
are considered as Gaussian for this purpose). It is called centred if E(y, X) = for 
every y E TL. Gaussian random variables are determined by their mean m = EX E TL 
and their covariance operator C: TL — ► TL defined by 

(y,Cx)=E((y,X-m)(X-m,x)). 

For details see e.g. |DPZ92 section 2.3.2]. The following lemma (see |DPZ92 propo- 
sition 2.15]) characterises the covariance operators of Gaussian measures. 

Lemma 2.1 Let X be a Gaussian random variable on a separable Hilbert space. Then 
the covariance operator C of X is self-adjoint, positive and trace class. 

A Gaussian random variable is said to be non-degenerate if (y, Cy) > for every 
y E TL \ {0}. An equivalent characterisation is that the law of (y,X) is a proper 
Gaussian measure (i.e. not a Dirac measure) for every y E TL \ {0}. Here we will 
always consider non-degenerate Gaussian measures. Then C is strictly positive definite 
and we can define C to be the inverse of —C. Since C is trace class, it is also bounded 
and thus the spectrum of C is bounded away from 0. 

We now construct an infinite dimensional process which, in equilibrium, samples 
from a prescribed Gaussian measure. Denote by w the cylindrical Wiener process 
on TL. Then one has formally 

oo 

w(t) = J2Pn(t)<l>n Vie (0,OO), (2.1) 
n=l 

where for n E N the (3 n are i.i.d. standard Brownian motions and <f> n are the (orthonor- 
mal) eigenvectors of C. Note that the sum 12. 1> does not converge in TL but that one 
can make sense of it by embedding TL into a larger Hilbert space in such a way that the 
embedding is Hilbert-Schmidt. The choice of this larger space does not affect any of 
the subsequent expressions (see also [DPZ92 1 for further details). 

Given C and C as above, consider the 7i-valued SDE given by II. 2\ . interpreted in 
the following way: 

x(t) = m + e ct {x(0)-m) + V2 [ e C(t ~ s) dw(s). (2.2) 
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If x € C([0, T],H) satisfies (E3 it is called a mz'/d solution of the SDE Oil . We have 
the following result. 

Lemma 2.2 Let C be the covariance operator and m the mean of a non- degenerate 
Gaussian random variable X on a separable Hilbert space TL. Then the corresponding 
evolution equation M.2X with C = — C _1 has continuous Ti-valued mild solutions. 
Furthermore, it has a unique invariant measure \xonTL which is Gaussian with mean m 
and covariance C and there exists a constant K such that for every initial condition 
.to G TL one has 

||law(x(Q) - /i|| TV <K{1 + ||x - m\\ H ) exp(-\\C\\^ H t), 

where \\ ■ |jxv denotes the total variation distance between measures. 

Pr oof. The existence of a continuous 7i-valued solution of the SDE II. 2> is established 
in |IMM + 90|. The uniqueness of the invariant measure and the convergence rate in the 
total variation distance follow by combining Theorems 6.3.3 and 7.1.1 from [DPZ96|. 
The characterisation of the invariant measure is established in [DPZ96 Thm 6.2.1]. 

□ 

We can both characterise the invariant measure, and explain the exponential rate of 
convergence to it, by using the Karhunen-Loeve expansion. In particular we give an 
heuristic argument which illustrates why Lemma l2~2l holds in the case m = 0: denote 
by (<^n)neN an orthonormal basis of eigenvectors of C and by (A„) ra6 N the correspond- 
ing eigenvalues. If X is centred it is possible to expand X as 

oo 

X = ^VXi^m (2-3) 

n=l 

for some real-valued random variables a n . (In contrast to the situation in d2. It the con- 
vergence in I2.3l l actually holds in L 2 (il, P, TL), where (f2, P) is the underlying prob- 
ability space.) A simple calculation shows that the coefficients a n are i.i.d. jV(0, 1) 
distributed random variables. The expansion (12. 31 is called the Karhunen-Loeve ex- 
pansion. Details about this construction can be found in [?]. 
Now express the solution x of II. 2> in the basis ((j> n ) as 

oo 

x(t) = ^2 7n(*)0n- 
n=l 

Then a formal calculation using 12. It and il.2\ leads to the SDE 

dr/n 1_ ^/2^" 

dt X n dt 

for the time evolution of the coefficients 7„ and hence j n is ergodic with stationary 
distribution Af(0, A„) for every n € N. Thus the stationary distribution of (II. 2t has the 
same Karhunen-Loeve expansion as the distribution of X and the two distributions are 
the same. 

In this article, the Hilbert space TL will always be the space L 2 ([0, 1], R d ) of square 
integrable Revalued functions and the Gaussian measures we consider will be distribu- 
tions of Gaussian processes. In this case the operatorC has a kernel C : [0, l] 2 — > M. dxd 
such that 

(Cx)(u)= / C(u,v)x(v)dv. (2.4) 
Jo 
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If the covariance function C is Holder continuous, then the Kolmogorov continuity 
criterion (see e.g. |DPZ92 Thm 3.3]) ensures that X is almost surely a continuous 
function from [0, 1] to M. d . In this case C is given by the formula 

C(u, v) = e((X(u) - m(u))(X(v) - m(¥))*) 

and the convergence of the expansion d2.3i is uniform with probability one. 

Remark 2.3 The solution of dl .21 may be viewed as the basis for an MCMC method 
for sampling from a given Gaussian process. The key to exploiting this fact is the 
identification of the operator C for a given Gaussian process. In the next section we 
show that, for a variety of linear SDEs, £ is a second order differential operator and 
hence (II .21 is a stochastic partial differential equation. If C has a Holder continuous 
kernel C, it follows from ( 12. 41 and the relation C = (— C)^ 1 that it suffices to find a 
differential operator C such that C(u, v) is the Green's function of — C. 



3 Conditioned Linear SDEs 

In this section we apply our sampling technique from section|2]to Gaussian measures 
which are given as the distributions of a number of conditioned linear SDEs. We con- 
dition on, in turn, a single known point ("subsection O.H . a single point with Gaussian 
distribution (subsection !3.2> and finally a bridge between two points (subsection !3.3> . 

Throughout we consider the Revalued SDE 

dX , dW 

—(u) = AX(u) + B—(u), VuG [0,1], (3.1) 
au du 

where A, B S M. dxd and W is the standard d-dimensional Brownian motion. We 
assume that the matrix BB* is invertible. We associate to ( 13. 1> the second order dif- 
ferential operator L formally given by 

L = (d u + A*)(BB*r 1 (d u - A). (3.2) 

When equipped with homogeneous boundary conditions through its domain of def- 
inition, we will denote the operator d3.2l > by L. We will always consider boundary 
conditions of the general form Dox(0) — and Dix(l) — 0, where Di = Aid u + hi 
are first-order differential operators. 

Remark 3.1 We will repeatedly write ]R d -valued SPDEs with inhomogeneous bound- 
ary conditions of the type 

d t x(t, u) = Lx(t, u) + g(u) + V2d t w(t, u) V(t, u) € (0, oo) x [0, 1], 

D o x(t,0) = a, D lX (t,Y) = b Vt€(0,oo), (3.3) 
ie(0, u) = x (u) Vu S [0, 1] 

where g : [0, 1] — > M. d is a function, d t w is space-time white noise, and a,b £ M d . We 
call a process x a solution of this SPDE if it solves J2.2I > with a;(0) = xq where C is L 
equipped with the boundary conditions Do/ (0) = and Dif(\) = 0, and m : [0, 1] — > 
M. d is the solution of the boundary value problem -Lin = g with boundary conditions 
Z?om(0) = a and Dira(l) = b. 

To understand the connection between d3.3i and 12.21 note that, if w is a smooth 
function, then the solutions of both equations coincide. 
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3.1 Fixed Left End-Point 

Consider the problem of sampling paths of J3 - 1 1 subject only to the initial condition 

X(0) = x~ S R d . (3.4) 
The solution of this SDE is a Gaussian process with mean 

m(u) = E{X(u)) = e uA x~ (3.5) 

and covariance function 

C (u,v) = e uA ( e- rA BB*e- rA dr)e vA (3.6) 



(see e.g. [KS91 section 5.6] for reference). Let C denote the differential operator L 
from (13. 2> with the domain of definition 

V{£) = {fe H 2 ([0, l],M d ) | /(0) = 0, ^-/(l) = A/(l)}. (3.7) 

Lemma 3.2 With L given by ( 15.21 one/ ( 13.71 f/ie function Co is the Green 's function 
for —C. That is 

LCq(u, v) — —5(u — v)I 

and 

C (0,«) = 0, d u C (l,v) = AC (l,v) Vug (0,1). 



Proof. From (13. 6> it is clear that the left-hand boundary condition Co(0, v) = is 
satisfied for all v e [0, 1]. It also follows that, for u ^ v, the kernel is differentiable 
with derivative 

anf s jAC Q (u,v) + BB*e- uA '^ A \ for u < v, and 
d u C {u,v) = I (3.8) 
I ACq(u, v) for u > v. 

Thus the kernel Co satisfies the boundary condition d u Co(l,v) = ACq(1,v) for all 

we [0,1). 

Equation J3.8i shows 

(BB*y\ du - A)C (u,v) = \l UA ** VA '> *™ < - d (3 .9) 

10 for it > u 

and thus we get 

LC (u,v)= (d u +A*)(BB*r 1 (d u -A)C (u,v) = Vu^v. 
Now let v G (0, 1). Then we get 

lim {BB*y\d u - A)C Q (u,v) = I 

and 

lim {BB*y l {d u - A)C Q {u, v) = 
This shows LCq(u, v) = —S(u — v)I for all v G (0, 1). □ 
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Now that we have identified the operator C = (— C) _1 we are in the situation 
of Lemma l2~2l and can derive an SPDE to sample paths of 13. II . subject to the initial 
condition J3.4I . We formulate this result precisely in the following theorem. 



Theorem 3.3 For every x G H the R d -valued SPDE 

d t x(t,u) = Lx(t,u) + V2d t w(t,u) V(i, it) G (0, oo) x (0, 1) (3.10a) 

x(f,0) = x~, d u x(t, 1) = Ax(t, 1) VtG(0,oo) (3.10b) 

x(0, u) = x {u) Vu G [0, 1] (3.10c) 



where d t w is space-time white noise has a unique mild solution. The SPDE is ergodic 
and in equilibrium samples paths of the SDE i l3.il ) with initial condition X(0) = x~. 

Proof. The solution of SDE J3 . 1 1 with initial condition (13.41 is a Gaussian process 
where the mean m is given by (13.51 . The mean m solves the boundary value prob- 
lem Lm{u) = for all u G (0, 1), m(0) = x~ and m'(l) = ylm(l). From Remark l3~T1 
we find that a: is a solution of the Hilbert space valued SDE (II. 2> for this function m. 

Lemma l3~2l shows that C, given by (13.21 with the boundary conditions from J3. lObl . 
is the inverse of — C where C is the covariance operator of the distribution we want to 
sample from (and with covariance function given by i3.6\ ). Lemma l2~2l then shows 
that the SPDE (13. 101 is ergodic and that its stationary distribution coincides with the 
distribution of solutions of the SDE (13 . 1 1 with initial condition X{0) = x~ . □ 

3.2 Gaussian Left End-Point 

An argument similar to the one in section lTTI deals with sampling paths of ( 13. 11 where 
X(0) is a Gaussian random variable distributed as 

X(0)~ JV(2T,£) (3.11) 

with an invertible covariance matrix E G R dxd and independent of the Brownian mo- 
tion W. 

Theorem 3.4 For every x £H the R d -valued SPDE 

d t x(t,u) = Lx(t,u) + V2d t w(t,u) V(i, u) G (0, oo) x (0, 1) (3.12a) 

d u x(t,0) = Ax(t,0) + BB*^ 1 ^- x~), d u x(t, 1) = Ax(t, 1) VtG(0,oo) 

(3.12b) 

i(0, u) = x (u) Vu G [0, 1] (3.12c) 

where dtw is space-time white noise has a unique mild solution. The SPDE is er- 
godic and in equilibrium samples paths of the SDE i li.il ) with Gaussian initial condi- 
tion \3.11\ . 

Proof. The solution X of SDE ( 13. 11 with initial condition ( 13. 1 1> is a Gaussian process 
with mean d3.5l > and covariance function 

C(u, v) = e uA Ze vA * + C (u, v), (3.13) 

where Co is the covariance function from ( I3.6l l for the case X(0) = (see Prob- 
lem 6.1 in Section 5.6 of IIKS91I for a reference). The mean m from ( 13.51 1 solves 
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the boundary value problem Lm(u) = for all u 6 (0,1) with boundary conditions 
to'(0) = Am(0) + BS*E- 1 (m(0) - x~) and m'(l) = Am(l). 

In order to identify the inverse of the covariance operator C we can use J3.8I > to find 



d u C(u,v) 



AC(u, v) + BB*eT uA \ vA * , for u < v, and 
AC'(u, v) for u > v 



and, since C(0, v) = E e 1 '' 4 , we get the boundary conditions 

<9„C(0, u) = AC(0, v) + BB^^CiO, v) 

and 

d u C(l,v) = AC(l,v). 
From (d u - A)e uA Y,e vA " = we also get 

LC(u, u) = Le^Ee^* + LC (u, v) = 

for all u ^ v and LC(u,v) — LCq(u,v) = —5(u,v)I for all u,v £ (0, 1). Thus 
C is again the Green's function for — C and the claim follows from Remark 1231 and 
Lemma l2~2l □ 



Remark 3.5 If A is negative-definite symmetric, then the solution X of SDE ( 13. 1> has 
a stationary distribution which is a centred Gaussian measure with covariance E = 
— \A~ X BB*. Choosing this distribution in ( I3.1H . the boundary condition J3. 12bl 
becomes 

d u x(t,0) = -Ax(t,0), d u x(t, 1) = Ax(t, 1) Vte(0,oo). 
3.3 Bridge Sampling 

In this section we apply our sampling method to sample from solutions of the linear 
SDE ( 13. H with fixed end-points, i.e. we sample from the distribution of X conditioned 
on 

X(0) = x~, X(l) = x + . (3.14) 

The conditional distribution transpires to be absolutely continuous with respect to the 
Brownian bridge measure satisfying d3 . 14-i . 

Let m and Cq be the mean and covariance of the unconditioned solution X of 
the SDE ( 13. li with initial condition X(0) = x~ . As we will show in Lemma l4~4l 
below, the solution conditioned on X(l) = x + is again a Gaussian process. The mean 
and covariance of the conditioned process can be found by conditioning the random 
variable (X(u), X{v), X{1)) for u < v < 1 on the value of X{1). Since this is a 
finite dimensional Gaussian random variable, mean and covariance of the conditional 
distribution can be explicitly calculated. The result for the mean is 

m(u) = m(u) + C (u, 1)C (1, lr 1 ^ - m(l)) (3.15) 

and for the covariance function we get 



C(u, v) = C (u, v) - C (u, l)Cb(l, lr^oCl, v). 



(3.16) 
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Theorem 3.6 For every x G H the R d -valued SPDE 

d t x = Lx + V2d t w V(i, u) G (0, oo) x (0, 1) (3.17a) 
sc(t,0) = ar~, x(t,l) = x + Vie(0,oo) (3.17b) 
ar(0, u) = x (u) Vu G [0, 1] (3. 17c) 

where dttv is white noise has a unique mild solution. The SPDE is ergodic and in 
equilibrium samples paths of the SDE \3.1\ subject to the bridge conditions \3.14\ . 

Proof. The solution of the SDE ( 13. II with boundary conditions 13.141 is a Gaussian 
process where the mean rh is given by 13. 151 and the covariance function C is given 
by 13. 161 . From formula d3.9l > we know LCq(u, 1) = and thus rh satisfies Lrh = 
Lm = 0. Since m(0) = x~ andm(t) = m(l) + C (l, 1)C (1, l)~ 1 (x+-m(l)) = x+, 
the mean rh solves the boundary value problem Lrh{u) = for all u G (0, 1) with 
boundary conditions m(0) = x~ and m(l) = x + . 

It remains to show that C is the Green's function for the operator L with homoge- 
neous Dirichlet boundary conditions: we have C(0, v) = 0, 

co., v) = Coo, v) - c o, i)c (i, D-^oa,^ = 

and using LCq(u, 1) = we find 

LC(u, v) = LCo(u, v) = —S(u — v)I. 
This completes the proof. □ 



4 The Kalman-Bucy Filter/Smoother 

Consider (ED with X replaced by the R m x Revalued process (X, Y) and A, B G 

R (m+n)x( m +n) chosen s0 as to obta j n the SDE 

d_ (X(u)\ (An 0\ (X(u)\ (B n \ d_ (W x (u)\ ™ 
du\Y(u)J \A 21 Oj \Y(u)J + V B 22 J du\W y (u)J- 

We impose the conditions 

x ~AT(aj-,A), y = o Op) 

and try to sample from paths of X given paths of Y. We derive an SPDE whose invari- 
ant measure is the conditional distribution of X given Y. Formally this SPDE is found 
by writing the SPDE for sampling from the solution (X, Y) of ( 14. 1> and considering the 
equation for the evolution of x, viewing y = Y as known. This leads to the following 
result. 

Theorem 4.1 Given a path Y sampled from H4.ll consider the SPDE 
d t x = ((d u + A^XBnB^yHdu - A n ))x 

+ A* 21 (B 22 B* 22 r 1 (^-~A 21 x^+V2d t w, (E!0 
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equipped with the inhomogeneous boundary conditions 

d u x(t, 0) = A n x(t, 0) + B ll Bl 1 h.- 1 (x{t, 0) - x~), 

d u x{t, 1) = A n x(t, 1) 0>) 

and initial condition 

x(0,u) = x (u) Vue [0,1]. (H3b l 

77;en /or every xo G 7i f/ie SPDE has a unique mild solution and is ergodic. Its 
stationary distribution coincides with the conditional distribution of X given Y for 
X, Y solving {OJ. 

The proof of this theorem is based on the following three lemmas concerning con- 
ditioned Gaussian processes. After deriving these three lemmas we give the proof of 
Theorem 14. II The section finishes with a direct proof that the mean of the invariant 
measure coincides with the standard algorithmic implementation of the Kalman-Bucy 
filter/smoother through forward/backward sweeps (this fact is implicit in Theorem l4. l> . 

Lemma 4.2 Let H = Hi H 2 be a separable Hilbert space with projectors Hi : H — > 

Hi- Let C ': H — > ri be a positive definite, bounded, linear, self-adjoint operator and 

denote Cij = HiCH*. Then Cn — C^C^C-n is positive definite and if Cn is trace 

_ i 

class then the operator Ci2C 22 2 is Hilbert- Schmidt. 
Proof. Since C is positive definite, one has 

2|(C 2 ia;,2/)| < (x,C n x) + (y,C 2 2y), 
for every (x, y) £ 7i. It follows that 

\(C2ix,y)\ 2 < (x,C n x)(y,C 2 2y), (4.3) 

and so 

\{C 21 x,C- 2 1/2 y)\ 2 < {x,C llX )\\y\\ 2 (4.4) 
for every y ^ in the range of C\l 2 . Equation J4.3b implies that C 2 \x is orthogonal to 

— 1/2 

kcr C 22 for every x 6 Hi. Therefore the operator C 22 C21 can be defined on all of Hi 
and thus is bounded. Taking y — C^^C^x in i4.4l gives ||C^ 2 1,/2 C2is|| 2 < (x,Cnx) 
and thus (x, (Cn — CuC^C^x) > for every x G Hi. This imphes that C 22 2 C 2 i 
and Ci 2 C 22 are both Hilbert-Schmidt, and completes the proof. □ 

Remark 4.3 Note that C being strictly positive definite is not sufficient to imply that 
Cn — Ci 2 C 22 C 2 i is also strictly positive definite. A counter-example can be constructed 
by considering the Wiener measure on H = L 2 ([0, 1]) with Hi being the linear space 
spanned by the constant function 1. 

Lemma 4.4 Let H = Hi © H 2 be a separable Hilbert space with projectors Hi: H ^ 
Hi. Let (Xi , X 2 ) be an H-valued Gaussian random variable with mean m = (mi , m 2 ) 
and positive definite covariance operator C and define Cij = HiCH*. Then the condi- 
tional distribution of Xi given X 2 is Gaussian with mean 

mi\ 2 = mi +Ci 2 C 22 (X 2 - m 2 ) (4.5) 



and covariance operator 

Ci\2 = Cn — Ci 2 C 22 C 2 i. 



(4.6) 
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Proof. Note that by Lemma lZTI the operator C is trace class. Thus C\\ and C22 are also 
trace class. Le 
inner product 



1 /2 

trace class. Let // be the law of X 2 and let Ho be the range of C 22 equipped with the 



(x,y) = (C 22 1/2 x,C 22 1/2 y). 

If we embed Ho <—* H2 via the trivial injection i( f) = f, then we find i*(f) — C22J '• 
Since i o i* = C22 is the covariance operator of /x, the space Ho is its reproducing 
kernel Hilbert space. From Lemma l4~2l we know that Ci 2 C 22 ^ 2 is Hilbert-Schmidt 
from H2 to H\ and hence bounded. Thus we can define 

A n /?— 1/2/1— 1/2 p p—l 

as a bounded operator from Ho to Hi . 

1/2 

Let (0 n )„ be an orthonormal basis of 7Y 2 . Then -0„ = C 22 4> n defines an orthonor- 
mal basis on Ho and we get 

Y H^nllwa = WCllC^ C^ 2 (j) n \\ 2 Hl = Y \\C\2C22 12 K\\ 2 Hl < °°' 

where the last inequality comes from Lemma R~2l This shows that the operator A is 
Hilbert-Schmidt on the reproducing kernel Hilbert space Ho- Theorem II. 3. 3 of |DF91 1 
shows that A can be extended in a measurable way to a subset of H 2 which has full 
measure, so that (14. 5> is well-defined. 

Now consider the process Y defined by 

YA _ (l Hl -A\ (Xx 
Y 2 J \0n 2 IhJ \X 2 

This process is also Gaussian, but with mean 



m 



I Hl -A\ f mA _ /mi - Am 2 
0h 2 Ih 2 ) V m 2/ V m 2 



and covariance operator 

r>Y _ ( Ifii ( C\\ C\ 2 \ f I-Hi 0« 2 \ _ / C11 — Ci2C 22 1 C2l 

" \0h 2 In J {C21 C22) \-A* I n J \ C 22 

This shows that Y\ = X\ — C\2C 22 X 2 and Y 2 = X 2 are uncorrected and thus inde- 
pendent. So we get 

E(Xj I X 2 ) = E(Xj - C 12 C 22 1 X 2 I X 2 ) + E(C 12 C 22 1 X 2 \ X 2 ) 
= E(-X"i — Ci2C 22 1 ^ 2 ) +Ci2C 22 1 ^2 
= mi — Ci 2 C 2 " 2 1 to 2 + C\2C 22 X 2 . 



This proves J4.5i and a similar calculation gives equality J4.6I . □ 

Remark 4.5 If we define as above C = (— C) _1 and formally define Cij = HiCU* 
(note that without additional information on the domain of C these operators may not 
be densely defined), then a simple formal calculation shows that mi|2 and C\\ 2 are 
expected to be given by 

mi| 2 = mi - C^C 12 {X 2 - m 2 ), Ci| 2 = -Cn- (4.7) 

We now justify these relations in a particular situation which is adapted to the case that 
will be considered in the remaining part of this section. 
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Lemma 4.6 Consider the setup of Lemma \4.4\a nd Remark \4.5\ and assume furthermore 
that the following properties are satisfied: 

a. The operator C can be extended to a closed operator C on WiD(C) Yl2D(£). 

b. Define the operators Cij — IT j CII*. Then, the operator Cn is self-adjoint and 
one has ker C\\ = {0}. 

c. The operator —C^ X L\% can be extended to a bounded operator from H2 into 
Hi. 

Then CyiC-2,2 can ^ e extended to a bounded operator from H2 into Hi and one has 
C12C22 1 = — £11 £l2' Furthermore, C21 maps Hi into the range of C22 and one has 

C xl x — (Cn — Ci2C 2 2 1 C2l)x, 

for every x G Hi. 

Proof. We first show that C12C22 1 = — £ii £12 • By property a. and the definition of C, 
we have the equality 

CTl^Cx + CU*Jl 2 Cx = -x (4.8) 

for every x G H, and thus CuCi^x = — C12C22X for every x S H2- It follows 
immediately that CuC^C^x = —C12X for every x S TZ(C22)- Since VXC22) is dense 
in H2, the statement follows from assumptions b. and c. 

Let us now turn to the second equality. By property a. the operator C21 maps Hi 
into the domain of £12 so that 

x = x - C12C21X + C12C21X = CuCnx + C12C21X, (4.9) 

for every x E Hi (the second equality follows from an argument similar to the one 
that yields ( 14. 8» . Since the operator C^ 1 is self-adjoint, we know from | Yos95 p. 195] 
that (CiaC^ 1 )* = C^C 21 . Since the left hand side operator is densely defined and 
bounded, its adjoint is defined on all of Hi, so that C21 maps Hi into the range of C22- 
It follows from i4.9\ that 



X = LiiCuX + Ci2C22C 2 2 1 C2lX, 

for every x G Hi. Using J4.8L this yields x — L\\C\\X — LuC^C^^ix, so that 
is an extension of Cn — Ci2C 22 1 C2i- Since both of these operators are self-adjoint, 
they must agree. □ 

Corollary 4.7 Let (X, Y) be Gaussian with covariance C and mean m on a separable 
Hilbert space H = Hi H2- Assume furthermore that C satisfies the assumptions of 
Lemmas \4.4\ and \4.6\ Then, the conditional law of X given Y is given by the invariant 
measure of the ergodic SPDE 

^ = Ciix-Cnllim + C^iY -n 2 m) + V2^, (4.10) 
at at 

where w is a cylindrical Wiener process on Hi and the operators Cij are defined as 
in Lemma W^l SPDE (14.101 is again interpreted in the mild sense (12.21 . 

Proof. Note that C^£ 12 can be extended to a bounded operator by assumption and 
the mild interpretation of J4. lQi is 

x t = M + e Cllt (x - M) + V2 f e Clllt - s) dw(s), (4. 1 1) 

Jo 
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with M = Him — £ 11 1 £i2(y — Tl 2 m). The result follows by combining Lemma l4~4l 
and Lemmal4~6lwith Lemma l2"2l □ 



These abstract results enable us to prove the main result of this section. 

Proof of Theorem l4.1\ Consider a solution (X, Y) to the SDE J4-. II . Introducing the 
shorthand notations 

Si = (Bn-B^) -1 , S 2 = (B 22 B* 22 )~ X , 



(L tl 


£i 2 \ 


(d u + A* n 


U21 


L 22 / 


■ I 



it follows by the techniques used in the proof of Theorem l3.4l that the operator £ cor- 
responding to its covariance is formally given by 

43A (B^B^ V 1 /9,-iii 
du)\ B 22 B* 22 ) \ -A 21 d u 

(d u + Al^Ou - A n ) - A* 2l V 2 A 21 A* 2X V 2 d % 
-d u Y, 2 A 2l d u T, 2 d u 

In order to identify its domain, we consider d3 . 1 2bl > with 

'A X 



" : \o r 

and we take the limit T — > 0. This leads to the boundary conditions 

d u x(Q) = A n x(0) + (AE 1 )- 1 (x(0) - x-), d u x(l) = A n x(l), 
2/(0) = 0, d u y(l) = A 21 x(l). 



Mm 



The domain of £ is thus H 2 ([0, l],R m x R"), equipped with the the homogeneous 
version of these boundary conditions. 

We now check that the conditions of Lemma 14.61 hold. Condition a. is readily 
verified, the operator £ being equipped with the boundary conditions 

d u x{$) = A 11 a;(0) + (AI] 1 )- 1 x(0), d u x{\) = A n x(l), __ 
2/(0) = 0, Ud u y(l) = Q, 

where II is the projection on the orthogonal complement of the range of ^21 ■ Note that 
the operator £ is closed, but no longer self-adjoint (unless A 2 i = 0). The operator C\\ 
is therefore given by 

£ u = (d u + A* n )E 1 (d a - A n ) - A* 21 E 2 A 21 , 

equipped with the boundary condition 

d u x(0) = Anx(0) + (AS 1 )- 1 a;(0), d u x(l) = A n x(l). 

It is clear that this operator is self-adjoint. The fact that its spectrum is bounded away 
from follows from the fact that the form domain of £ contains n^llil?(£) and that 
there is a c > with (a, Ca) < — c\\a\\ 2 for all a E T>(£). Thus condition b. holds. 

The operator £12 is given by the first-order differential operator A 21 Y, 2 d u whose 
domain is given by functions with square-integrable second derivative that vanish at 
0. Since the kernel of £ xl has a square-integrable derivative, it is easy to check that 
£n £12 extends to a bounded operator on H, so that condition c. is also verified. 
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We can therefore apply Lemma 14761 and Lemma l272l The formulation of the equa- 
tion with inhomogeneous boundary conditions is an immediate consequence of Re- 
mark l3.ll a short calculation to remove the inhomogeneity in the boundary condi- 
tions (I4.2bl and change the inhomogeneity in the PDE (I4.2hl shows that (14. 2> can be 
written in the form (14.101 or ( 14. Ill with the desired value for M , the conditional mean. 
Since L\\ is indeed the conditional covariance operator, the proof is complete. □ 

Remark 4.8 For Y solving ( 14. U the derivative 4^ only exists in a distributional sense 
(it is in the Sobolev space H~ 1 / 2 ~ t for every e > 0). But the definition \2.2\ of a 
mild solution which we use here applies the inverse of the second order differential 
operator C\\ to 4^, resulting in an element of iJ 3 / 2 ~ £ in the solution. 

Remark 4.9 Denote by x(t, u) a solution of the SPDE ( 14. 2\ and write the mean as 
x(t, u) — Kx(t, u). Then, as t — ► oo, x(t, u) converges to its limit x(u) strongly in 
L 2 ([0, l],M. m ) and x(u) must coincide with the Kalman-Bucy filter/smoother. This 
follows from the fact that x equals ¥.(X | Y). It is instructive to demonstrate this result 
directly and so we do so. 

The mean x(u) of the invariant measure of ( 14. 2\ satisfies the linear two point bound- 
ary value problem 



(-T + AlOiBnB*,)- 1 ^ - A n )x(u) 
du du 

+ A* 21 (B 22 B^r 1 ( ^ - A 21 x(u)) =0 Vu e (0, 1) 



(4.13a) 

- A n , = n Vi; <= en n 

V du 

4-^(0) = Anx(0) + B u B?iA _1 (£(0) - x"), (4.13b) 
aw 

4-2(1) = Aiis(l). (4.13c) 
du 

The standard implementation of the Kalman filter is to calculate the conditional 
expectation X(u) = K(X(u) \ Y(v), < v < u) by solving the initial value problem 

4-5(«) = A u 5(u) + S^MIj - S^JA^B^By^^iSdi) + BnBJj 
du 

5(0) = A (4.14) 

and 

4 W = (^n - S(u)A* 21 (B 22 B* 22 r 1 A 21 )X + S{u)A* 21 {B 22 B* 22 T l( ^ 
du du 

X(0) = x-. (4.15) 

The Kalman smoother X, designed to find X(u) = E(X(u) \ Y(v), < v < 1), is 
then given by the backward sweep 

— X(u) = A n X(u) + BnBl 1 S(uy 1 (X(u) - X(u)) Vu e (0, 1) 
du 

X(1) = X(1). (4.16) 

See |0ks98 section 6.3 and exercise 6.6] for a reference. We wish to demonstrate that 
x(u) = X(u). 
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Equation (14. 16i evaluated for u = 1 gives equation ( 14. 13ci . When evaluating I4.16> 
at it = we can use the boundary conditions from J4. 14i and (14. 15i to get equa- 
tion (14. 1 3bi . Thus it remains to show that X(u) satisfies equation (14. 13al . We proceed 
as follows: equation (14. 16i gives 



and so 



(4- + A^XBnB^r 1 ^ - A U )X 
du du 

= (4- + A* u )(B n Bt 1 )- 1 B 11 Bt 1 S- 1 (X - X) 
du 

^(±+Al 1 )S~ 1 (X-X) 
du 



(l+A^BnBt^-A^X 



(4.17) 



We have 



= A* x 5 + —S- 1 )(X-X) + S-' — iX - X). 
du ) du 



d s- 1 = -s-'^-s- 1 



du du 
and hence, using equation I4.14> . we get 

4- S- 1 = -S^An - A^S- 1 + A* 21 (B 22 B* 22 r 1 A 21 - S^BuB^r 1 . (4.18) 
du 

Subtracting (14. 15i from J4. 16i leads to 

5- ^iX -X) = S- l Axx(X -X) + S- 1 B u Bt 1 S- 1 (X - X) 

du 

-A* 21 (B 22 B* 22 r 1 (^-A 21 xy (4.19) 
By substituting ( 14. 1 81 . I4.19> into ( 14.171 1 and collecting all the terms we find 

i± + AWBnBti-\± - A n) X = -A^B^ (f - A 21 X 

which is equation ( I4.13a^ 

We note in passing that equations 14.141 1 to ( 14.161 1 constitute a factorisation of the 
two-point boundary value problem (14. 1 31 reminiscent of a continuous LU-factorisation 
of £n. 

5 Numerical Approximation of the SPDEs and Sampling 

A primary objective when introducing SPDEs in this paper, and in the nonlinear com- 
panion |HSV)> is to construct MCMC methods to sample conditioned diffusions. In 
this section we illustrate briefly how this can be implemented. 

If we discretise the SDE II. 21 in time by the ^-method, we obtain the following 
implicitly defined mapping from (x k , £ fe ) to x*\ 

^ir" = { 9Cx * + (1 - 6)Cxk ) - Cm + \/l^ fc ' 
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where £ fc is a sequence of i.i.d Gaussian random variables in H with covariance oper- 
ator / (i.e. white noise in in Ti). The Markov chain implied by the map is well-defined 
on Ti for every 9 € [h > 1]. This Markov chain can be used as a proposal distribution for 
an MCMC method, using the Metropolis-Hastings criterion to accept or reject steps. 
To make a practical algorithm it is necessary to discretise in the Hilbert space Ti, as 
well as in time t. This idea extends to nonlinear problems. 

Straightforward calculation using the Karhunen-Loeve expansion, similar to the 
calculations following Lemma lZ2l shows that the invariant measure of the SPDE i ll .2t 
is preserved if the SPDE is replaced by 

dx /— - dw _ 
— = - x + m+ V2C— . (5.1) 
at at 

Such pre-conditioning of Langevin equations can be beneficial algorithmically because 
it equalises convergence rates in different modes. This in turn allows for optimisation 
of the time-step choice for a Metropolis-Hastings algorithm across all modes simulta- 
neously. We illustrate this issue for the linear Gaussian processes of interest here. 

Equation i5.1\ can be discretised in time by the ^-method to obtain the following 
implicitly defined mapping from (x k , £ fc ) into x*: 



At 



Yftc*+(i 



Now £ fc is a sequence of i.i.d. Gaussian random variables in Ti with covariance oper- 
ator C. Again, this leads to a well-defined Markov chain on Ti for every 9 € [3,1]. 
Furthermore the invariant measure is C/(l + (9 — \)At). Thus the choice 6 = | has a 
particular advantage: it preserves the exact invariant measure, for all At > 0. (These 
observations can be justified by using the Karhunen-Loeve expansion). Note that 

1 + 6 At) x* = (l - (1 - 9) At) x k 

When 9 = |, choosing At = 2 generates independent random variables which there- 
fore sample the invariant measure independently. This illustrates in a simple Gaussian 
setting the fact that it is possible to choose a globally optimal time-step for the MCMC 
method. To make a practical algorithm it is necessary to discretise in the Hilbert space 
Ti, as well as in time t. The ideas provide useful insight into nonlinear problems. 



6 Conclusions 

In this text we derived and exploited a method to construct linear SPDEs which have 
a prescribed Gaussian measure as their stationary distribution. The fundamental rela- 
tion between the diffusion operator C in the SPDE and the covariance operator C of 
the Gaussian measure is C — (— C) _1 and, using this, we showed that the kernel of 
the covariance operator (the covariance function) is the Green's functions for L. We 
illustrated this technique by constructing SPDEs which sample from the distributions 
of linear SDEs conditioned on several different types of observations. 

These abstract Gaussian results were used to produce some interesting results about 
the structure of the Kalman-Bucy filter/smoother. Connections were also made be- 
tween discretisations of the resulting SPDEs and MCMC methods for the Gaussian 
processes of interest. 
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In the companion article |HSV| we build on the present analysis to extend this 
technique beyond the linear case. There we consider conditioned SDEs where the drift 
is a gradient (or more generally a linear function plus a gradient). The resulting SPDEs 
can be derived from the SPDEs in the present text by the addition of an extra drift term 
to account for the additional gradient. The stationary distributions of the new nonlinear 
SPDEs are identified by calculating their Radon-Nikodym derivative with respect to 
the corresponding stationary distributions of the linear equations as identified in the 
present article; this is achieved via the Girsanov transformation. 

The Girsanov transformation is used to study the connection between SPDEs and 
bridge processes in |RVE05|; it is also used to study Gibbs measures on R in [?]. 
However the results concerning bridges in this paper are not a linear subcase of those 
papers because we consider non-symmetric drifts (which are hence not gradient) and 
covariance of the noise which is not proportional to the identity. Furthermore the non- 
linear results in |HSV| include the results stated in |RVE05 1 as a subset, both because 
of the form of the nonlinearity and noise, and because of the wide-ranging forms of 
conditioning that we consider. 
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